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Abstract 

A d-dimensional nonparametric additive regression model with de- 
pendent observations is considered. Using the marginal integration 
technique and wavelets methodology, we develop a new adaptive esti- 
mator for a component of the additive regression function. Its asymp- 
totic properties are investigated via the minimax approach under the 
L2 risk over Besov balls. We prove that it attains a sharp rate of con- 
vergence which turns to be the one obtained in the i.i.d. case for the 
standard univariate regression estimation problem. 
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1 Introduction 



1.1 Problem statement 

Let d be a positive integer, (yj,Xi)jgz be a M x [0, 1] '^-valued stationary 
process on a probability space {Q, A, P) and p be a given real measurable 
function. The unknown regression function associated to (l^,Xj)jg2 and p 
is defined by 

g(x) = E(p(y)|X = x), X = (xi, ...,Xd)e [0, 1]^. 

In the additive regression model, the function g is considered to have an 
additive structure, i.e. there exist d unknown real measurable functions 
gi, . . . ,g(l and an unknown real number such that 

d 

g{x.) = p + J^aeixe)- (1-1) 
For any i € our goal is to estimate gi from n observations 

(Yi,Xi), . . . , (Yn,^n) of {Yi,Xi)i(=z- 



1.2 Overview of previous work 



When (Yi,X.i)i^z is a i.i.d. process, this additive regression model becomes 



the standard one. In such a case. Stone in a series of papers [27|, |28|, |29 | 
proved that g can be estimated with the same rate of estimation error as 
in the one-dimensional case. The estimation of the component gi has been 
investigated in several papers via various methods fkerriel, splines, wavelets, 
etc.). See e.g. fl, Q, Q, 0,0, Q, B, S, H, Q and Q. 

In some applications, the i.i.d. assumption on the observations is too 
stringent. For this reason, some authors have explored the estimation of 
g£ in the dependent case. When (Yi, XAgz is a strongly mixing process, 
this problem has been addressed by [5[, [9|, and results for continuous time 
processes under a strong mixing condition have been obtained by 10|, 
In particular, they have developed non-adaptive kernel estimators for gi and 
studied its asymptotic properties. 
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1.3 Contributions 



To the best of our knowledge, adaptive estimation of g£ for dependent pro- 
cesses has been addressed only by [l^]- The lack of results for adaptive 
estimation in this context motivates this work. To reach our goal, as in 



32l | , we combine the marginal integration technique introduced by |22] with 



wavelet methods. We capitalize on wavelets to construct an adaptive thresh- 
olding estimator and show that it attains sharp rates of convergence under 
mild assumptions on the smoothness of the unknown function. By adaptive, 
it is meant that the parameters of the estimator do not depend on the pa- 
rameter (s) of the dependent process nor on those of the smoothness class of 
the function. In particular, this leads to a simple and easily implementable 
estimator. 

More precisely, our wavelet estimator is based on term-by-term hard 
thresholding. The idea of this estimator is simple: (i) we estimate the 
unknown wavelet coefficients of gi based on the observations; (ii) then we 
select the greatest ones and ignore the others; (iii) and finally we reconstruct 
the function estimate from the chosen wavelet coefficients on the considered 
wavelet basis. Adopting the minimax point of view under the L2 risk, we 
prove that our adaptive estimator attains a sharp rate of convergence over 
Besov balls which capture a variety of smoothness features in a function 
including spatially inhomogeneous behavior. The attained rate corresponds 
to the optimal one in the i.i.d. case for the univariate regression estimation 
problem (up to an extra logarithmic term). 



1.4 Paper organization 

The rest of the paper is organized as follows. Section [2] presents our as- 
sumptions on the model. In Section [3l we describe wavelet bases on [0,1], 
Besov balls and tensor product wavelet bases on [0, 1]*^. Our wavelet hard 
thresholding estimator is detailed in Section HI Its rate of convergence under 
the L2 risk over Besov balls is established in Section \E\ Section [6] provides 
a discussion of the relation of our result with respect to prior work. The 
proofs are detailed in Section [71 
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2 Notations and assumptions 



In this work, we assume the following on our model: 



Assumptions on the variables. 

• For any z G {1, . . . , n}, we set Xj = {Xi^i, . . . , Xd^i). We suppose that 

— for any i G {1, . . . , n}, Xi^i, . . . , X^^i are identically distributed 
with the common distribution Zi([0, 1]), 

— Xi, . . . ,X„ are identically distributed with the common known 
density /. 

• We suppose that the following identifiability condition is satisfied: for 
any i £ {1, . . . ,d} and i £ {1, . . . , n}, we have 

H9eiXe,i)) = 0. (2.1) 



Strongly mixing assumption. Throughout this work, we use the strong 
mixing dependence structure on (yi,Xi)igz- For any m € Z, we define the 
m-th strongly mixing coefficient of (yi,Xj)jgz by 

am= sup \F{AnB) -F{A)¥{B)\, (2.2) 

(yx.) 

where ^_^o cr-algebra generated by . . . , (Yli, X_i), (Yo^ Xq) and 

(Y'K.) 

is the cr-algebra generated by (Y^, X^), (Ym+i, X^+i), • • • . 
We suppose that there exist two constants 7 > and c > such that, 
for any integer m > 1, 

"m < 7exp(— cm). (2.3) 

^'urther details on strongly mixing dependence can be found in 3, 0, Q, 
2^ and fl. 
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Boundedness assumptions. 

• We suppose that p € Li(M) nLoo(K), i.e. there exist constants Ci > 
and C2 (supposed known) such that 

r^\p{y)\dy<C,, (2.4) 
and supj^gK \p{y)\ < (72. (2.5) 

• We suppose that there exists a known constant c > such that 

inf /(x)>c. (2.6) 

xe[o,i]'* 

• For any m G {1, . . . , n}, let /(yo,Xo,y™,x„) be the density of (Yq, Xq, X^), 
/(yo,Xo) tlie density of (Yq, Xq) and, for any (y, x, y*, x*) € M x [0, l]'^ x 

M X [0,1]'^, 

hm(,y y* T — 

/{yo,Xo,y™,X^)(y>x,y*,x,) - /(yo,Xo)(2/,x)/(yj,^Xo)(2/*>x*). 

(2.7) 

We suppose that there exists a known constant C > such that 

sup sup |/im(2/,x,y^,x*)| < C. (2.8) 

m£{l,...,n} {j/,x,j/.,x,)gRx[0,1]'^xRx[0,1]<^ 

Such boundedness assumptions are standard for the estimation of g£ from a 



strongly mixing process. See e.g. (lOl. 



3 Wavelets and Besov balls 

3.1 Wavelet bases on [0, 1] 

Let i? be a positive integer. We consider an orthonormal wavelet basis 
generated by dilations and translations of the scaling and wavelet functions 
<j) and V from the Daubechies family dh2R- In particular, (p and ip have 
compact supports and unit L2-norm, and ip has R vanishing moments, i.e. 
for any r G {0, . . . , R — 1}, f x^^{x)dx = 0. 



5 



Define the scaled and translated version of (p and ip 

(t)j,k{x) = 2^/^^{2^x - k), 4^j,kix) = 2^/^ip{2^x - k). 

Then, with an appropriate treatment at the boundaries, there exists an 
integer r satisfying 2'^ > 2R such that, for any integer > r, the collection 

A: G {0, . . . , 2> -1}; j S N-{0, . . . , j*-!}. A; G {0, . . . , 2^-1}}, 

is an orthonormal basis of L2([0, 1]) = {h : [0, 1] — ?• M; h?{x)dx < oo}. 
See BQ. 

Consequently, for any integer > r, any /i G L2([0, 1]) can be expanded 
into a wavelet series as 

2^*-! oo 2J-1 

^(^) = X] aj,^k(pj,,k{x) + X] X] Phkiphkix)., X G [0, 1], 
fc=o j=i* fe=o 

where 



ttj.fc = / h{x)(t)j^k{x)dx, j3j^k = / h{x)^j^k{x)dx. 
Jo Jo 



(3.1) 



3.2 Besov balls 



As is traditional in the wavelet estimation literature, we will investigate the 
performance of our estimator by assuming that the unknown function to be 
estimated belongs to a Besov ball. The Besov norm for a function can be 
related to a sequence space norm on its wavelet coefficients. More precisely, 
let M > 0, s G (0, i?), p > 1 and q>l. A function h in L2([0, 1]) belongs to 
Bp ,j(M) if, and only if, there exists a constant M* > (depending on M) 
such that the associated wavelet coefficients (|3.ip satisfy 

In this expression, s is a smoothness parameter and p and q are norm pa- 
rameters. Besov spaces include many traditional smoothness spaces. For 
particular choices of s, p and q, Besov balls contain the standard Holder 
and Sobolev balls. See [20|. 
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3.3 Wavelet tensor product bases on [0, 1]*^ 

For the purpose of this paper, we will use compactly supported tensor prod- 
uct wavelet bases on [0, l]'^ based on the Daubechics family. Let us briefly 
recall their construction. For any x = (xi, . . . ,Xd) G [0, l]*^, we construct a 
scaling function 

d 

$(x) = []<^(x,) , 

v=l 

and 2*^ — 1 wavelet functions 



i^ixu) JJ H^v) when n G {1, . . . , d}, 



v=l 



Yl iix„) Y[ <j){x^) when u G {d + 1, . . . , 2<^ - 1}, 

, veAu v^Au 



where (^■u)ue{d+i,...,2d-i} forms the set of all non void subsets of {1, ... , d} 
of cardinality greater or equal to 2. 

For any integer j and any k = (fci , . . . , A;^) , define the translated and 
dilated versions of $ and as 

k(x) = 2^''^/2$(2^xi - fci, . . . , 2^xd - kd), 
^i,k,n(x) = 2^'^/2*„(2^xi -h,..., 2^Xd - kd), for any € {1, . . . , 2^ - 1}. 

Let Dj = {0, . . . , 2-^ — l}*^. Then, with an appropriate treatment at the 
boundaries, there exists an integer r such that the collection 

{^r,k, k € Dr-, (^'j,k,n)„g{l,...,2''-l}> j G N - {0, . . . , T - 1}, k G Dj} 

forms an orthonormal basis of L2([0, l]'') = {h : [0, l]*^ — >■ M; /i^(x)(ix < 

oo}. 

For any integer such that j* > r, a function h G L2([0, l]'') can be 
expanded into a wavelet series as 

2'*-! oo 

/i(x) = ^ aj,,k$,,,k(x) ^i,M*i,k,«(x), X G [0, 1]^ 

keDj» u=l j=j, keDj 
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where 

Oj-.k = / /i(x)$j,k(x)dx, /3j,k,« = / /i(x)^'j,k,«(x)dx. (3.2) 

J[0,1]'* -'[0,1]'' 

4 The estimator 

4.1 Wavelet coefRcients estimator 

The following proposition provides a wavelet decomposition of gi based on 



the "marginal integration" method (introduced by [23] ) and a tensor product 
wavelet basis on [0, 1]''. 

Proposition 4.1 Suppose that ()2.ip holds. Then, for any j^, > t and £ G 
{1, . . . , d}, we can write 

23* -1 oo 2J-1 

gi{x) = ^ aj,^k/(t>j,A^) + X] X] ^hk/tpj^kix) - /i, X G [0, 1], 

k=l j=jt k=l 

where 



aj,k,i - dj. 



"j,k/ — Oj^kt,. 



= 2-^('^-i)/2 / 5(x) V k(x)dx, (4.1) 
2-^('^-i)/2 / 5(x) ^.•,k/(x)dx, (4.2) 



and = (/ci, . . . , fc^.i, /c^+i, . . . , /c^) anc? D* = {0, . . . , 2^' - 1}'^ ^ 

Remark 4.1 Due to the definitions of g and properties of ^'j,k/) ^j,k/ is 
nothing hut the wavelet coefficient of g^, i.e. 

bj,k,e = / gt{x)tpj^k{x)dx = f3j^k- (4.3) 
Jo 

Proposition 14.11 suggests that a first step to estimate gi should consist in 
estimating the unknown coefficients aj^k,e (|4-ip and 6j,fc/ ()4.2p . To this end, 
we propose the following coefficients estimators 

a,., = a,.,, = 2-('^-)/^iX:7iT E (4.4) 

™ i=i J^^i) k_,eD* 
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and 

hk,e = h,,, = 2-^^'-'y'l-±J^ v,^,,XX.). (4.5) 

1 = 1 ■' ^ k_<;GD' 

These estimators enjoy powerful statistical properties. Some of them are 
collected in the following propositions. 

Proposition 4.2 (Unbiasedness) Suppose that (j2.ip holds. For any j > 
T, £ e {1, . . . ,d} and k e {0, . . . ,2^ - 1}, aj^k/ and bj^k,e in and ([i3]) 

are unbiased estimators of aj^k,i and bj^k/ respectively. 

Proposition 4.3 (Moment inequality I) Suppose that the assumptions 
of Section d hold. Let j > t such that 2^ < n, k G {0, . . . , 2-^' - 1}, i G 
{1, . . . ,d}. Then there exists a constant C > such that 

E {{aj^k,i - a,,k/?) < E ((6,- fc,^ - b,^k/f^ < C^. 

Remark 4.2 In the proof of Proposition \4-3[ we only need to have the ex- 
istence of two constants C > and q £ (0, 1) such that Ylm=i ^'^'^m < C* < 
oo. This latter inequality is obviously satisfied by (|2.3p . 

Proposition 4.4 (Moment inequality II) Under the same assumptions 
of Proposition \4-3[ there exists a constant C > such that 

Proposition 4.5 (Concentration inequality) Suppose that the assump- 
tions of Section\^hold. Let j > t such that 2^ < nj (In rif' , k G {0, . . . , 2-' — 
1}, I G and A„ = (Inn/n)"*^/^. Then there exist two constants 

C > and k > such that 

P {Kk/ - bj,k,e\ > 'tA„/2) < C^. 
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4.2 Hard thresholding estimator 

We now turn to the estimator of gi from aj^k/ and bj^k/ as introduced in 
(j4.4p and (|4.5p . Towards this goal, we will only keep the significant wavelet 
coefficients that are above a certain threshold according to the hard thresh- 
olding rule, and then reconstruct from these coefficients. In a compact form, 
this reads 

2^-1 ji 2J-1 

9e{x) = ar,kAr,k{x) + ^i.^'^^{|fe,,fe,d>«A„}V'i,fc(a;) - /I, (4.6) 

k=0 j=T k=0 

where 

1 " 

In , ji is the resolution level satisfying 2^'^ = [n/(lnn)^], k is a large enough 
constant (the one in Proposition 14. 5p and 



/Inn 

An = \ • 

V n 

Note that, due to the assumptions on the model, our wavelet hard thresh- 
olding estimator (|4.2p is simpler than the one of 32 1 . 



5 Minimax upper-bound result 

Theorem 15.11 below investigates the minimax rates of convergence attained 
by gg over Besov balls under the L2 risk. 

Theorem 5.1 Leti G {1, . . . ,d}. Suppose that the assumptions of Section\^ 
hold. Letgi be the estimator given in ()4.2p . Suppose that gg € Bp g(M) with 
g > 1, > 2 and s G (0,i?)} or {p G [1,2) and s G {l/p,R)}. Then there 
exists a constant C > such that 

E(^j\Ux)-ge{x)fdx^ <C 

The proof of Theorem 15.11 is based on a suitable decomposition of the L2 
risk and the statistical properties of (j4.4p and (j4.5p summarized in Propo- 
sitions 1121 1131 m and [13] above. 
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6 Relation to prior work 

The rate (lnn/n)^*/*^^'*+^) is, up to an extra logarithmic term, known to 
be the optimal one for the standard one-dimensional regression model with 
uniform random design. See e.g. [l^ and 30 1. 

Theorem 15.11 provides an "adaptive contribution" to the results of 
and 10|, lll[- Furthermore, if we confine ourselves to the i.i.d. case. 



we recover a similar result to 



32l . Theorem 3] but without the condition 
s > ma.x(d/2, d/p). The price to pay is more restrictive assumptions on the 
model {p is bounded from above, the density of X is known, etc.). Addition- 
ally, our estimator has a more straightforward and friendly implementation 
than the one in 32 1. 



7 Proofs 

In this section, the quantity C denotes any constant that does not depend 
on j, k and n. Its value may change from one term to another and may 
depends on (p ot ^. 

7.1 Technical results on wavelets 

Proof of Proposition l4.ll Because of (|2.5p . we have g G L2([0, 1]^). For 
any > r, we can expand g on our wavelet-tensor product b 

2''-! oo 

ff(x) = ^ a,-.,k^>,k(x) +^Y.Y1 /5j-,M^i,k,«(x), X G [0, l]t7.1) 

keDj^ u=l j=j, keDj 

where 

"i.k = / g(x)$j,k(x)dx, /?j,k,M = / 5'(x)^j,k,«(x)dx. 

J[0,1]'* -'[0,1]'* 

Moreover, using the "marginal integration" method based on (j2.ip . we 
can write 

„ d 

ge{xe) = / ^(x) TT dx^ - x<?g[0, 1]. (7.2) 
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Since (t)j^k{x)dx = 2 ■'/^ and '4)j^k{x)dx = 0, observe that 

~ d 

^[o,i]<*-i ^Ji 

and 

J[o,i]'*-i ^,=1 [ otherwise. 

Therefore, putting (j7.ip in (j7.2p and writing x = x^, we obtain 

oo 

Or, equivalently, 

2^*-! oo 2^-1 

k=i j=j, k=i 



where 

and 

bj,k,e = bj^kfj 



i = 2-j('^-^)/2 / ^^(x) V cD^- k(x)dx 



2-^('^-i)/2 / 5(x) V ^,,k/(x)dx. 
Proposition 14.11 is proved. 



Proposition 7.1 For any £ G {1, . . . , d}, j > t and k = ke ^ {0, . . . ,2^ 

1}, set 

hfl{^)= E '^.>(x), /.g(x)= vl/.kXx), xG[0,l]'^. 

k_£GD* k_fGD* 
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Then there exists a constant C > such that, for any a £ {1,2}, 

sup \hfli^)\ < C2^''l\ [ |/ifi(x)|dx < C2-^/^2^(d-m 

FfO.lld ^f0.11<* 



xG[0,l]'' J [0,1] 

and 

(/i(;)(x))2dx = 2^-(^-i). 



[0,1]'* 



Proof: 



Since sup^.g[o^i] \(l)j,k{x)\ < Cl^l'^ and sup^gjo^i] ELo^ l</'j,fc(a^)l < (^2^/2, 
we obtain 



2^-1 ' 



sup |/i.'i(x)| = ( sup sup <C2^''^/2. 

Using |(/<j^fc(x)|(ix = C2~-^/^, we obtain 



2^-1 „i 



0j,A,.(x)[(ix 



• Since, for any (uk)kGDj , /[o,i]d (EkeD, ""k^i.kCx)] (ix = EkeD.^L 
we obtain 

2 



/ ih^i.)fd.= [ ( 



«>,-k(x)| dx = 2^('^-i). 



Proceeding in a similar fashion, using sup^g[o,i] \'^j,k{x)\ < C2-^/^, 
Jo \i^j,ki^)\dx = C2-J/2 and, for any (nk)keDj, /[o,i]d (EkeDj ^k*i,k,f(x 
EkeDj ^k' obtain the same results for h^^^. 

This ends the proof of Proposition 17.11 
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7.2 Statistical properties of the coefficients estimators 
Proof of Proposition 14.21 We have 



p{Yi)_ 
I 

k_,e-D 



S-^'^'^-^)/' / 5(x) V <i>,,k(x)dx = a,- fc,,. 



Proceeding in a similar fashion, we prove that K{bj^k,£) = bj^k,. 



Proof of Proposition [4T3l For the sake of simphcity, for any i € {1, , 

set 



Thanks to Proposition we have 

E ((%, fc,, - a,- = V(%, fe,,) = 2-^('^"^)-32 V ^E • 
An elementary covariance decomposition gives 

(n \ n v—\ 

Y^ZA = nV(Zi) + 2^ J^Cov(Z,,Z„) 
i=l / t;=2 u=l 

n v—1 

< nV(Zi) + 2' 



j;^Cov(Z, 



v=2 u=l 
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Using ()2.5p . (j2.6p and Proposition 17. H we have 



infxe[o,i]'' / (x) \ /(Xi) I ^ 



C7 



/ TTTl E '^^>(x)| /(x)(ix 

[ I c|>,,k(x)| dx = C2^('^-i). (7.5) 



It follows from the stationarity of (li,Xj)jg2 and 2^ < n that 

n -u— 1 



(n — m)Cov (Zq, Z„ 



m=l 



<Ri + R2, (7.6) 



where 

Ri=n'^\Cov{Zo,Zm)\, i?2 = n^ |Cov (Zq, Z^)] . 

m=l ni=2J 

It remains to bound Ri and R2. 

(i) Bound for Ri. Let, for any (y,x,y*,x,) e M x [0, 1]"^ x M x [0,1]"^, 
/i„(y,x,y^,x*) be ([22]). Using ([TH]) . ([23]) and Proposition [7H we 
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obtain 



I Gov (Zo, Z„ 



/i,„(y,x,2/*,x*) X 



oo J[0,1]'* J-oo J\Q,\Y 

^^^^ E ^i,k(x)^ E 'J>j,k(x.) I dydxdy.dx. 



/(x 



< I / / |/im(y,x,y*,x*)| X 

-oo J[0,1]'' J-oo J[0,l]d 



piy) 



/(x) 



E ^.-.kCx: 



< c 

Therefore 



k_^GD* 



E ^i,k(x*: 



dydxdy^dx^ 



[0,1]" 



E ^^-.kCx) 



k_fGD* 



(7.7) 



(ii) Bound for R2. By the Davydov inequahty for strongly mixing pro- 
cesses (see 0]), for any g G (0, 1), we have 

l-q 



|Cov(Zo,Z^)|<10a^ E lZo|2/(i-'?) 



< lOal ( .^"P^-^"'^^„"A sup 



I infxe[o,i]d /(x) xG[o,i]'* 
By (|2.5p . (j2.6p and Proposition 17.11 we have 

suPygK \p{y)\ 

—r 77^ sup 

™ixe[o,i]'' /ixj xe[o.i]'' 



E '^^■.k(x) 



2g 



(E(Z2)) 



2\\ 



E '^^■.k(x) 


< C sup 


E '^^■.k(x) 




xe[o,i]'' 


k_,eD* 



< C2^'^/2_ 



By (j7.5p . we have 



E(Z2) < C2^('^-i). 
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Therefore 

|Cov(Zo,Z„)| <C72«2^('^~i)a^. 
Observe that X^J^^i m'^am = j'^ Z^m=i rn'^exp{—cqm) < oo. Hence 

n n 
m=2i m=2i 

Putting (US]), (IZ21) and together, we have 

n v—1 



v=2 u=l 

Combining ([73]), (USD, (IZ3|) and dZS]), we obtain 



E 



Proceeding in a similar fashion, we prove that 



This ends the proof of Proposition 14.3 



(7.9) 



Proof of Proposition 14.41 It follows from (12. Sp . (12. and Proposition 
Othat 



Ih I <r 9-i('i-i)/2 ^ Sr^ \p0^i)\ 



. r ff ^ sup 

inixe[o,i]d / W xe[o,i]<* 



E *J-.k/(x) 
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Because of (|2.5p . we have supxg[o,i]d < C. It follows from Proposition 

Othat 



Hence 



\bj,kA < 2-^-('^-i)/2 / |5(x)| 
'[o,ip 



I] 'J'i,k/(x) 



l^i.fc/ — ftj,A;/| < \bj,k,e\ + \bj,k,i\ < C2-'/^. 
It follows from (17. lip and Proposition 14.31 that 



E - bj,k,er < C^2^E - bj^k/Y )<C-. 



n 



The proof of Proposition 14.41 is complete. 



(7.10) 



(7.11) 



Proof of Proposition 14.51 Let us first state a Bernstein inequality for 
exponentially strongly mixing process. 
! — ' 

Lemma 7.1 ([17]) Lei 7 > 0, c > and {Yi)i^z be a stationary process 
with the m-th strongly mixing coefficient Um (12. 2p . Let n he a positive in- 
teger, /i : M ^ C he a measurahle function and, for any i ^ "L, Ui = h{Yi). 
We assume that E([/i) = and there exists a constant M > satisfying 
\Ui \ < M . Then, for any m € {1, . . . , [?^/2]} and A > 0, we have 



i=l 



> A < 4 exp 



M 

W{Dm/m + XMm/3)J +32^""™, 



where Dm = max^gii 2^} V (^XlLi 

We now apply this lemma by setting for any i G {1, . . . ,n}, 
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Then we can write 



i=l 



So 

where C/i, . . . , C/„ are identicaUy distributed, depend on (Yj, Xj)j£z satisfying 

• by Proposition 14.21 we have IE(f/i) = 0, 

• using arguments similar to Proposition 14.31 with / instead of n, we 
prove that 



it) 



< CI. 

I 



Hence Dm = max^gji 2m} 
• proceeding in a similar fashion to (j7.1ip . we obtain \Ui \ < C2-'/^. 

Lemma |7. 1 1 applied with the random variables Ui, . . . , Un, A = kA„/2, = 
(Inn/n)-^/^, m = nlnn with n > (chosen later), M = C2-'/^, 2^ < n/(lnn)^ 



and (|2.3p gives 

< L exp — C- r — + — nexpf— cm 

^/ / K^lnn \ 2^/2 ^ ^ ^ 

< (7 exp —C -7- , ,„ + — — TTT-nexpf— cnmn) 

y l + Kn2J/2lnn(lnn/n)V2y (lnn/n)i/2 ^ 

Therefore, for large enough k and u, we have 

P (l^j.fc,^ - bj,k,i\ > kA„/2) < C^. 
This ends the proof of Proposition 14.51 
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7.3 Proof of Theorem 

Using Proposition 14.11 we have 

9e{x) - gi{x) 

2^-1 31 2J-1 

k=0 j=T k=0 

oo 2J-1 

- XI X] bj,kii'jM^) - - A*)- 

i=ii+l fc=0 

Using the elementary inequahty: (x + y)^ < 2(x^ + 2/^), € M^, and the 

orthonormahty of the wavelet basis, we have 

E [^j\Mx) - ge{x)fdx^ <2{T + U + V + W), (7.12) 

where 

2^-1 



k=0 

ji 2J-1 oo 2J-1 



j=T k=0 J=ii+1 fc=o 



(i) Bound for T. We proceed as in the proof of Proposition 14.31 By 
()2.ip . we have E(/)(Yi)) = /u. Thanks to the stationarity of {Yi)i^z, we 
have 



T = V(/I) < -YipiY,)) + 2- X |Cov {p(Yo),piYm))\ . 

m=l 

Using ()2.5p . the Davydov inequality (see f^) and ()2.3p . we obtain 

1 / " \ 1 n2s/(2s+1) 

T<-C\U^Y.<)^^cUc{^^) . (7.13, 

\ m=l / ^ ' 



(ii) Bound for U . Using Proposition 14.31 we obtain 



1 /lnn\2^/(2s+^) 
U <C2^-<C[—\ . (7.14) 

n \ n I 
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(iii) Bound for W. For q > I and p > 2, we have gi € Bp,j(M) C 
Bl^{M). Hence, by 

i=ii+i V / V " / 

For g > 1 and p € [1,2), we have 5^ G B^g{M) C 52,t<!^^~^^^(^)- 
Since s > 1/p, we have s + 1/2 - 1/p > s/{2s + 1). So, by (03]), 

00 

W < C ^ 2-2i(s+l/2-l/p) < (-,2-2ji(s+l/2-l/p) 

i=ii+i 

/n \3\ 2(s+l/2-l/p) /, X 2s/(2s+l) 

< ^ / (Inn)n < c ( ] 

Hence, for g > 1, {p > 2 and s > 0} or {p € [1,2) and s > 1/p}, we 
have 

/hin\^'/(^'+^) 

VF<C( — ) . (7.15) 



n / 

(iv) Bound for V. We have 

F = Vi + y2 + V3 + F4, (7.16) 

where 

j=T k=0 
31 2^-1 



j=r k=0 

^3 = Z Z ^ (^>.^^{|b,,.,.|<KA„}l{|fe,,fe,,|>2«A4) 



j=r fc=0 

and 



^4 = Z Z ^ ri^/^{|6,,fe,l|<«A„}l{|fe,,fc,,|<2«A4) • 

j=T fc=0 
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Bounds for Vi and V3. The following inclusions hold: 

jl^'i.fc/j < i^K, \bj,k,£\ > 2kA„| C - bj±/\ > kA„/2|, 

||&i,fc,£| > kA„, \bj^k,e\ < C ^\bj^k/ - bj^k/\ > kA„/2| 

and < kA„, \bj^k,i\ > 2kA„| C |!6j,fc,<?| < 2\bj^k,£ - 



So 

ii 2^-1 

max(yi, V3) <CJ2Y.^ - bj,k,ifl 

j=T k=0 



{\bj,k,l-bj,k,l\>K.>^n/2} 



Applying the Cauchy-Schwarz inequality and using Propositions 
3] and 2^ < n, we have 



1/2 



E ( (bj,k/ - bj,k,e) l{|b,,,,,-b,,,,,|>«A„/2} 

1/2 



< (e (^(bj,k/ - bj,k/)^)) (P {\bj^k,e - bj,k/\ > 



< C 
Therefore 



2 A 
n J 



1 



1/2 



< C7- 



1 



n 



2 ■ 



1 1 1 /, n2s/(2s+1) 

max(yi, Vs) < C4 y 2^' < C42^^i < C- < C — (7.17) 

Bound for V2. Using Proposition 14.31 we obtain 
E ({bjAe - bjAif) < C- < C—. 



Hence 



1 jl 2J-1 



Jl Z / Z / {|fej,fc,£|>KA„/2}' 

j = T fe=0 



Let j2 be the integer defined by 

n \ 1/(2^+1)' 



2J2 



Inn/ 



(7.18) 



We have 



V2<^2,1+V2,2, 
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where 

Inn 

j=T k=0 

and 

Inn ^'"^ 

1^2,2 = C— ^ l{|6,,fc,,|>KA„/2}- 
i=i2+l A:=0 

We have 

V2 1 < C V 2^ < C 2^2 < 

n n \ 11 I 

For g > 1 and p > 2, we have ge G B^^^{M) C B|^^(M). So, by 

. ji 2^-1 00 2J-1 

" i=j2+i fc=o i=i2+i fc=o 

^/lnn\2^/(2.+i) 

~ \ n 

For g > 1, p € [1,2) and s > 1/p, using , ,|>«a„/2} < 

C|6j,fc/|P/A^ = C|/3j- fclVA^ and (2s + 1)(2 - p)/2 + (s + 1/2 - 
l/p)p = 2s, we have 

" i=i2+i fc=o ^ ^ i=i2+i 

/l„„\(2-p)/2 /, n2s/(2s+1) 



n y V ^ / 

So, for r > 1, {|j > 2 and s > 0} or {j5 G [1, 2) and s > 1/f}, we 
have 

/lnn\2'/(2'+^) 
F2<C — . (7.19) 



n y 



Bound for V4. We have 



ii 2^-1 

J=T fc = 
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Let j2 be the integer (|7.18p . Then 



Va < ^4,1 + ^4,2, 

where 

j'z 2^-1 ji 2J-1 

^4,1 = ^Y1 ^lkA{\b,,k,,\<2KX„}^ ^4,2 = ^ Yl ^lkA{\bj,k,l\<2^K}- 
j=T k=0 i=j2+l k=0 

We have 

j,=r j=r 

For g > 1 and p > 2, we have € B^ g{M) C B| „^(M). Hence, 
by (1131), 



~ 2J-1 /, n2s/(2s+1) 

J=J2+1 A:=0 ^ ^ 



For g > 1, p G [1, 2) and s > 1/p, using g^D, ^'fc/l{|6,-,,,,|<2«A„} < 
CA^^|6,- = CA^P|/3,- and (2s + 1)(2 - p)/2 + (s + 1/2 - 
= 2s, we have 

ji 2^-1 . ^ (2-p)/2 ji 2J-1 

^4,2 < cxi-^ E Ei/3.>r = ^(^) E 

j=J2+l A:=0 ^ ^ 3=32+1 k=0 



< C ( — E 2-^("+i/2-i/p)p 

V / i=j2 + l 

/l„„\(2-p)/2 /, n2s/(2s+1) 



n J \ n J 

Thus, for q > 1, {p > 2 and s > 0} or {p G [1)2) and s > 1/p}, 
we have 

V^<C(^\ . (7.20) 
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It follows from (f7l6]l . (fTTT]) . fTTO]) and (r7:20]l that 

V<C{^) . (7.21) 

Combining (I71^ . (I71^ . (miD . (TTTS]) and (USB), we have, for g > 1, 
> 2 and s > 0} or {p e [1, 2) and s > 1/p}, 

E ^ iUx) - 9i(.x)fdxj < C (^— J 
The proof of Theorem 15.11 is complete. 
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